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IMAGE DATA PROCESSING METHOD AND CORRESPONDING DEVICE 



Field of the Invention 

The present invention relates to a processing method 
5 and, more particularly, to an image data processing 
method receiving an input image data splitted into 
elementary unit of information to be used in embedded 
applications . 

The present invention also relates to an image data 
10 processing device implementing such image data 
processing method. 

Background of the Invention 

As is well known in the technical field of image 
processing, during its life an image is processed by a 
15 plurality of electronic devices, that create, acquire, 
display store, read and write the image itself. 

The image data processing device, and the corresponding 
processing method deal with an image acquired by means 
of an image acquisition device, for example a scanner. 

20 The image data so obtained are usually organized into a 
raster of pixels, each pixels providing an elementary 
image inf ormat ion . 

In other words, images are, at the most basic level, 
arrays of digital values, where a value is a collection 
25 of numbers describing the attributes of a pixel in the 
image. For example, in bitmaps, the above mentioned 
values are single binary digits. 

Often, these numbers are fixed-point representation of 
a range of real number; for example, the integers 0 
3 0 through 255 are often used to represent the numbers 
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from 0.0 to 1.0. Often too, these numbers represent the 
intensity at a point of the image (gray scale) or the 
intensity of one color component at that point . 

An important distinction has to be made in the images 
5 to be processed between achromatic and colored images. 

In fact, achromatic light has only one attribute, which 
is the quantity of light. This attribute can be 
discussed in the physic sense of energy, in which case 
the terms intensity and luminance are used, or in the 
10 psychological sense of perceived intensity, in which 
case the term brightness is used. 

It is useful to associate a scale with different 
intensity levels, for instance defining 0 as black and 
1 as white; intensity levels between 0 and 1 represent 
15 different levels of grays. 

The visual sensations caused by colored light are much 
more richer than those caused by achromatic light. 
Discussion on color perception usually involves three 
quantities, known as: hue, saturation and lightness. 

20 1. Hue distinguishes among colors such as red, green, 
purple and yellow. 

2. Saturation refers to how far a color is from a 
gray of equal intensity. Red is highly saturated; pink 
is relatively unsaturated; royal blue is highly 

25 saturated; sky blue is relatively unsaturated. Pastel 
colors are relatively unsaturated; unsaturated colors 
include more white light than do the vivid, saturated 
colors . 

3. Lightness embodies the achromatic notion of 
30 perceived intensity of a reflecting object. 

A fourth term, brightness, is used instead of lightness 
to refer to the perceived intensity of a self-luminous 
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object (i.e. an object emitting rather than reflecting 
light) , such as a light bulb, the sun or a CRT. 

The above mentioned features of colors seem to be 
subjective: they depend on human observers' judgment. 
5 In reality, the branch of physics known as colorimetry 
provides for an objective and quantitative way of 
specifying colors, which can be correlated to the above 
perceptual classification . 

A color can be represented by means of its dominant 
10 wavelength, which corresponds to the perceptual notion 
of hue; excitation purity corresponds to the saturation 
of the color; luminance is the amount or intensity of 
light. The excitation purity of a colored light is the 
proportion of pure light of the dominant wavelength and 
15 of white light needed to define the color. 

A completely pure color is 100 % saturated and thus 
contains no white light, whereas mixtures of a pure 
color and white light have saturations somewhere 
between 0 and 100 %. White light and hence gray are 0 % 
20 saturated, contains no color of any dominant 
wavelength . 

Furthermore, light is fundamentally electromagnetic 
energy in the 4 00-700 nm wavelength part of the 
spectrum, which is perceived as the colors from violet 
25 through indigo, blue, green, yellow and orange to red. 
The amount of energy present at each wavelength is 
represented by a spectral energy distribution P(l), as 
shown in figure 1. 

The visual effect of any spectral distribution can be 
3 0 described by means of three values, i.e. the dominant 
wavelength, the excitation purity, and the luminance. 
Figure 2 shows the spectral distribution of figure 1, 
illustrating such three value. In particular, it should 
be noted that at the dominant wavelength there is a 
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spike of energy of level e2 . White light, the uniform 
distribution of energy level el is also present. 

The excitation purity depends on the relation between 
el and e2 : when el=e2, excitation purity is 0 %; when 
5 el=0, excitation purity is 100 %. 

Luminance, which is proportional to the integral of the 
area under such curve, depends on both el and e2 . 

A color model is a specification of a 3D color 
coordinate system and a visible subset in the 
10 coordinate system within which all colors in a 
particular range lie. For instance, the RGB (red, 
green, blue) color model is the unit cube subset of a 
3D Cartesian coordinate system, as shown in figure 3. 

More specifically, three hardware-oriented color models 
15 are RGB, used with color CRT monitors, YIQ, i.e. the 
broadcast TV color system that is a re-coding of RGB 
transmission efficiency and for downward compatibility 
with black and white television and CMY (cyan, magenta, 
yellow) for some color-printing devices. Unfortunately 
20 none of these models are particularly easy to use 
because they do not relate directly to intuitive color 
notions of hue, saturation, and brightness. Therefore, 
another class of models has been developed with ease of 
use as a goal, such as the HSV (hue, saturation, value) 
25 - sometimes called HSB (hue, saturation, brightness) , 
HLS (hue, lightness, saturation) and HVC (hue, value, 
chroma ) mode 1 s . 

With each model is also given a means of converting to 
some other specification. 

3 0 As stated above, the RGB color model used in color CRT 
monitors and color raster graphics employs a Cartesian 
coordinate system. The RGB primaries are additive 
primaries; that is the individual contributions of each 



4 



1 

60980084-2 

primary are added together to yield the result. The 
main diagonal of the cube, with equal amounts of each 
primary, represents the gray levels: black is (0,0,0); 
white is (1,1,1). 

5 Following such gray line implies the change of the 
three Cartesian value R, G and B at the same time, as 
shown with a point-dotted line in figure 4A; this 
situation weights the computational charge of the image 
processing steps requiring the individuation of gray 
10 regions. 

The RGB model is hardware -oriented . By contrast HSV (as 
well as HSB or HLC) model is user-oriented, being based 
on the intuitive appeal of the artist's tint, shade, 
and tone. The coordinate system is cylindrical, as 
15 shown in figure 4B. 

The HSV model (like the HLC model) is easy to use. The 
grays all have S=0 and they can be removed from an 
image data raster by means of a cylindrical filter in 
proximity of the V axes, as shown in figure 5; 
20 moreover, the maximally saturated hues are at S = l, 
L=0.5. 

The HLS color model is a reduced model obtained from 
the HSV cylindrical model, as shown in figure 6; the 
reduction of the color space is due to the fact that 

2 5 some colors cannot be saturated. Such space subset is 

defined is a hexcone or six-sided pyramid, as shown in 
figure 7. The top of the hexcone corresponds to V=l 
which contains the relatively bright colors. The colors 
of the V=l plane are not all of the same perceived 

3 0 brightness however. 

Hue or H, is measured by the angle around the vertical 
axis with red at 0° green at 120° and so on (see figure 
7) , Complementary colors in the HSV hexcone are 18 0° 
opposite one another. The value of S is a ratio ranging 
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from 0 on the center line (V axis) to 1 on the 
triangular sides of the hexcone. 

The hexcone is one unit high in V, with the apex at the 
origin. The point at the apex is black and has a V 
5 coordinate of 0 . At this point, the values of H and S 
are irrelevant. The point S=0, V=l is white. 
Intermediate values of V or S=0 (on the center line) 
are the grays. It is therefore immediately apparent the 
simplicity of use of the HSV or equivalent color space 
10 in order to obtain the gray regions. 

Adding a white pigment corresponds to decreasing S 
(without changing V) . Shades are created by keeping S=l 
and decreasing V. Tones are created by decreasing both 
S and V. Of course, changing H corresponds to selecting 
15 the pure pigment with which to start. Thus, H, S, and V 
correspond to concepts from the perceptive color 
system. 

The top of the HSV hexcone corresponds to the 
projection seen by looking along the principal diagonal 

2 0 of the RGB color cube from white toward black, as shown 

in figure 8 . 

In figure 9 is shown the HLS color model, which is 
defined in a double -hexcone subset of the cylindrical 
space. Hue is the angle around the vertical axis of the 
25 double hexcone, with red at 0°. The colors occur around 
the perimeter: red, yellow, green, cyan, blue and 
magenta. The HLS space can be considerated as a 
deformation of HSV space, in which white is pulled 
upward to form the upper hexcone from the V=l plane. As 

3 0 with the single -hexcone model, the complement of any 

hue is located 180° farther around the double hexcone, 

and saturation is measured radially from the vertical 

axis form 0 on the axis to 1 on the surface . Lightness 

is 0 for black (at the lower tip of the double hexcone) 

35 to 1 for white (at the upper tip) . 
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Many hardware and software packages are currently 
available in the technical field of the electronic 
image processing which provide for image data 
processing methods and corresponding devices. However, 
5 it should be noted that the Applicant is not aware of 
any package capable of operating in both the personal 
computer/work station field as well as in the embedded 
devices field. 

In fact, the embedded devices have a plurality of needs 
10 which turn into tight limitations for the image 
processing devices themselves. Particularly, the image 
processing in an embedded environment seeks : 

to reduce the size of the image data in order to 
limit the memory area employed by the image data 
15 processing devices; 

to increase the amount of any text portion 
comprised in a document that can be OCR' able, i.e. it 
should be possible to acquire and understand such 
portion by means of an Optical Characters Recognitor 
20 (OCR) ; 

to get as final result of the image data 
processing device an image viewable and printable, 
which is close to the original acquired image. 

Known document analysis that tried to fit the above 
25 requirements have the problem of being computationally 
very heavy and not suited for embedded applications 
where processing power and memory requirements are 
stringent and important . 

So, even if these solutions may perform an acceptable 
3 0 analysis of the document, they are not applicable in an 
embedded environment . 

The main purpose of the known document analysis is the 

extraction of features and the classification of text 
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and images in the analyzed documents. Examples of 
analysis used in this technical field are known from 
the publication "Document Image Analysis" to L. 
O 1 Gorman and R. Kasturi, IEEE Computer Society Press, 
5 which is a collection of all the most relevant papers 
regarding document analysis. 

All the known approaches deal with the recognition of 
different types of areas on a page. The areas are 
normally classified into regions of text, photo and 
10 line art. The page is then divided into these different 
areas (normally in a mutually exclusive way) and each 
is treated in a different way. In other terms, the 
known document analysis deal with understanding the 
"type" of information that is on the page. 

15 . These solutions tend to sub-divide the page into 
mutually exclusive regions that contain different type 
of information. 

Other known devices deal with decomposed documents, 
i.e. documents translated into a plurality of 
20 elementary image information called pixels. Such 
devices provide a treatment of the decomposed document 
as a whole, or at least are able to reconstruct the 
information they need from the input document format. 

An illustrative and not limiting example is a BW fax 
25 machine. If such device can deal only with BW data and 
the document contains a mixture of sparse color and BW 
data, the fax machine image processing device must be 
able to reconstruct a single BW page from the pieces of 
the decomposed original document . 

3 0 A known way to comply with the embedded environment 
requirements leads to peripheral devices that support 
only the specified features of a particular product; 
that is how cost and performance are satisfied. 
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An object of the present invention is to teach how to 
build into the system the necessary architecture which 
allows for image processing functionality not fixed or 
predetermined a priori. In this way, the functionality 
5 is not static and can be alterated just like a program 
running on a personal computer or on a work station. 

Starting from the image processing devices basic 
requirements, several known solution packages propose a 
"plug and play" capability of components. 
10 Unfortunately, these known solutions do not offer the 
speed or flexibility in order to operate within an 
embedded environment . 

Moreover, all of the known processing devices suffer 
large execution footprints, large memory needs, and a 
15 code data space that exceeds the ties of the embedded 
application. 

So, another object of the present invention is that of 
figuring how to provide the same level of user 
configurability and customization, without violating 
20 the strict requirements of the embedded market. 

Summary of the Invention 

The solution idea behind this invention is that of 
isolating the means whereon data are transported around 
the system. 

25 More particularly, the present invention describe a 
dynamic image data processing pipeline which is 
distributed in a transparent fashion, not influencing 
any other component within the system using the 
processing device according to the present invention. 

3 0 It should be noted that, unlike a series of plugs 

components or a series of small executables that are 

chained together to form the pipeline, the present 

solution focuses in making small atomic operations that 
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function independent of any context. 

According to this solution idea, the invention relates 
to an image data processing method as defined in the 
enclosed claim 1 . 

The invention also relates to an image data processing 
device, implementing such method, as defined in the 
enclosed claim 12 . 

The features and advantages of the image data 
processing method and device according to the invention 
will be appreciated by the following description of a 
preferred embodiment given by way of non-limiting 
examples with reference to the annexed drawings . 

Brief Description of the Drawings 

Figure 1 shows an example of a spectral energy 
distribution of a color; 

Figure 2 shows the spectral distribution of figure 1, 
illustrating dominant wavelength, excitation purity and 
luminance . 

Figure 3 shows the 3D Cartesian representation of the 
RGB color space, with the fundamental colors; 

Figure 4A shows the RGB color space of figure 3 and the 
gray line within; 

Figure 4B shows the cylindrical representation of the 
HSV/HLC color space; 

Figure 5 shows a gray filter for the HSV/HLC color 
space ; 

Figure 6 shows the HLS color space; 

Figure 7 shows the single -hexcone representation of HSV 
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color space; 

Figure 8 shows a section of figure 7; 

Figure 9 shows the double -hexcone representation of the 
HSV color space; 

Figure 10 shows schematically the image processing 
method according to the present invention; 

Figure 11 shows schematically examples of path for the 
method according to the present invention; 

Figures 12A, 12B and 12C show an example of atomic 
operation used in the method according to the present 
invention and its implementation; 

Figures 13A, 13B and 13C show another example of atomic 
operation used in the method according to the present 
invention and its result; 

Figures 14A, 14B and 14C show another example of atomic 
operation used in the method according to the present 
invention and its implementation; 

Figure 15 shows a particular result for the atomic 
operation of figures 14A, 14B and 14C; 

Figure 16 shows another example of atomic operation 
used in the method according to the present invention; 

Figure 17 shows another example of atomic operation 
used in the method according to the present invention; 

Figures 18A and 18B show another example of atomic 
operation used in the method according to the present 
invention and its implementation; 

Figure 19A shows schematically and functionally an 
image data processing device according to the present 
invention; 
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Figure 19B shows more particularly a detail of figure 
19A. 

Figure 20 shows more particularly a PDF application of 
the method according to the present invention, 

5 Detailed Description of the Invention 

The basic idea underlying the present application is 
that of providing a processing method reducing the 
complexity of image processing in an undetermined 
fashion. The idea provides also for implementing a 
10 corresponding configurable plug- &-playable embedded 
firmware solution . 

The image data processing method according to the 
present invention comprises the following steps : 

1. Identification of a final device which receives 
15 the output data, in terms of its requirements. 

2. Get a color image input data a pixels raster 
format . 

3. Select a group of basic operations to be performed 
on said image data. 

20 4. Create a sequence that links and ordinate said 
basic operations. 

5 . Apply the above predetermined sequence to the 
input image data. 

6. Create a plurality of layers of organized meta- 
25 level data. 

7. Assemble at least an output layer which is optimal 
for the identified device. 

8. Send each assembled output layer to the identified 
device . 
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Examples of final devices may be printers, copiers, 
image systems, office applications (such as, word 
processors, spreadsheet, . . . ) , facsimiles, OCR 

applications, PDF applications or the like. 

5 Each of these final devices has specific requirements. 
For example, high resolution data having no color 
information are required in order to perform an OCR 
function; instead, such color information should be 
contained in an output image data for a color printer. 

10 It should be noted that in the above steps 3 to 5 each 
sequence yields to an unique result. Moreover, the 
steps 6 and 7 specify, for each layer, which kind of 
data to process and how to process such data. Finally, 
step 9 creates an output layer in the specific format 

15 required by the identified device. 

In other terms, the processing method according to the 
present invention provide a plurality of basic 
functions for the digital image pixels and the 
combination between such functions in order to obtain 
20 the desired layer, as shown in figure 10. 

According to this figure, an input data 1, having, for 
example, the form of a raster of pixels, is processed 
according to a first function fl in order to obtain a 
plurality of layers 2, corresponding to different 
25 representations of the processed image data. 

Moreover, the method according to the present invention 
comprises a second function f2 that select/create the 
desired format for the output data 3 . 

Figure 11 shows schematically examples for image data 
3 0 paths according to the proposed image data processing 
method. Particularly, in figure 11 the method according 
to the present invention is shown in terms of pipelines 
4, i.e. in term of "paths" of atomic operations to be 
performed on the input data 1 in order to obtain a 
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particular output format 3 . More particularly, figure 
11 shows two different output representation, LAYER 1 
and LAYER 2 , obtained by means of a first, PATH1, and a 
second data path, PATH 2 . 

5 Image based segmentation and analysis involves the 
classification of unique data features in input data. 
This data is viewed and processed as separate entities, 
without any relationship between the features and the 
input image being maintained. 

10 The present invention expands upon the traditional 
processing viewpoint of image segmentation through the 
introduction of meta- level representation that 
corresponds to the relationship between layers. Layers 
become a logical meta- level viewport or window into the 

15 input data. 

For example, a single input data source contains all 
content for the image, but carries no meta-level 
information as to how that data can be extracted, 
viewed or manipulated. For example, if the input image 
is requested for OCR 1 able content, the baseline image 
is incapable of delivering the data necessary for the 
OCR engine. If the input image is requested for color 
image data content, the baseline input image is unable 
to extract the necessary information that conveys the 
image content only. This knowledge is traditionally 
maintained in the algorithms or hard coded application; 
it is not flexible and independent of the application 
itself . 

Indeed, layers carry a cognitive or meta-level 

knowledge that allows for the mapping mechanism 

necessary to create and generate the proper data view 

required by each layer. .This concept moves from a 

single flat raster view to a rich multi-layer view 

where each layer has a different perspective upon the 

underlying baseline layer, and can be processed 
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independent or dependent upon other layers. These 
layers can be defined to process output optimal for a 
destination device if so designed and specified. 

Take for example the process of creating two 
5 overlapping images from a single flat raster structure. 
One could conceive of a requirement where image content 
that is monochrome is to be processed mutually 
exclusive of image content that is color. Furthermore, 
the two different images are to be processed with 

10 different resolutions, pixel bit depth, compression 
methods, to mention a few. Traditional methods would 
separate the two processes, generating different 
intermediates, and later figuring out how to combine 
and weave the intermediates into a single artefact for 

15 some destination or other purpose. 

With layers, one can conceive a meta-level view that 
sees the flat input raster image one of the components 
of a 5 layer representation. The first layer is the 
baseline input source image. The second layer is the 
20 knowledge necessary to create the perspective of 
monochrome content only. The third layer is the meta- 
level information necessary to create the view for 
color data content. The fourth layer contains the 
knowledge for the relationship between layers 2 & 3 and 

2 5 how they are related mutually exclusive of each other . 

The final layer captures the perspective of how the 
data is extracted from each appropriate layer to create 
the final representation optimal for the output device. 

This differs from traditional means, whom view the 
30 source data as a flat structure and the relationship 
between data is captured in the application or 
hardcoded into a specific algorithm. Given layers, into 
the baseline flat image source data, one can extract 
any given portion of a layer. From layers, one can 

3 5 obtain meaningful content that may or may not have any 
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resemblance to the original source, but has meta-level 
connection between the layers that create the final 
perspective for how that data is to be processed and 
represented at the output destination in a flexible 
5 fashion. 

The pipelines or paths define the sequence of atomic 
operations to be performed on the input image data 1. 
Such atomic operations, which are individually known in 
the field of image data processing, can be grouped 
10 together to generate a plurality of IP (Image 
Processing) tools . 

The IP tools comprise: 

a transformation of an image pixel from the RGB 
format to another image space format, for example the 
15 HLS (or HLN, for hue, lightness, chroma indicator N) 
format ; 

a grouping function that associates elementary 
unit of information in order to obtain an unique 
information group to be processed, as the blobbing 
2 0 technique; 

scalars processes that change the image 
dimensions ; 

a down sampling function; 

a thresholding function; 

25 - a split and merge data function; 

encoder functions ; 

an AND /OR and other data extraction function; 

compression functions (for example, the G4 or JPEG 
compression method) ; 
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a half-tone approximation; 

data format readers and writers; 

a re-mapping or stretching function; 

a filtering function. 

5 In figure 12A is shown the image process of re-mapping, 
highly used in the field of photos processing. Since 
the content of data of an image histogram is usually 
concentrated approximately in the first 60% portion of 
such image histogram, the information contained in such 
10 portion are distributed or re-scaled over the all image 
data space in order to increase the useful information 
data . 

Figure 12B shows schematically an implementation of the 
above re-mapping process, called re-mapping 

15 architecture 5. The re-mapping architecture 5 comprises 
a RGB to HLS converter 6 , connected to an HLS to RGB 
converter 9, by means of a series of an HLS filter 7 
and a stretch block 8 . 

More precisely, the re-mapping process deals only with 
20 the L channel, i.e. the lightness information, as shown 
in figure 12C, in which a re-mapping architecture 5' 
comprises a RGB to HLS converter 6' having an H, L, and 
S output channels. The L channel is connected to series 
of an HLS filter 7' and a stretch block 8'. The re- 
25 mapping architecture 5' further comprises an HLS to RGB 
converter 9' receiving the H and S channel as outputted 
from the RGB to HLS converter 6 1 and the L channel as 
processed by means of the HLS filter 7 1 and the stretch 
block 8 1 . 

3 0 The halftone approximation is shown in figures 13 A to 
13C. It is pointed out that many displays and hardcopy 
devices are bilevel - they produce just two intensity 
levels - and even 2 - or 3 bit -per-pixel raster 
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displays produce fewer intensity levels than desired. 
The range of available intensities can be expanded on 
the basis of the spatial integration that human's eyes 
perform. If anyone views a very small area from a 
5 sufficiently large viewing distance, his eyes average 
fine detail within the small area and record only the 
overall intensity of the area. 

This phenomenon is exploited in printing black-and- 
white photographs in newspapers, magazines, and books, 
in the technique called halftoning (also called 
"cluster-dot ordered dither" in computer graphics) . 
Each small resolution unit is imprinted with a circle 
of black ink whose area is proportional to the 
blackness 1-1 (where I is the intensity) of the area in 
the original photograph. Figure 13A shows part of a 
halftone pattern, greatly enlarged. Note that the 
pattern makes a 45° angle with the horizontal, called 
the screen angle. 

Graphics output devices can approximate the variable- 
area circles of halftone reproduction. For example, a 2 
* 2 pixel area of a bilevel display can be used to 
produce five different intensity levels at the cost of 
halving the spatial resolution along each axis. The 
patterns shown in Fig. 13B can be used to fill the 2*2 
areas with the number of ON pixel that is proportional 
to the desired intensity. Figure 13C shows a face 
digitized as a 351*351 image array and displayed with 
2*2 patterns . 

Another image data processing comprises a down sampling 
30 and thresholding of the data. In particular, as shown 
in figure 14A, once the colorfulness of each pixel has 
been calculated and extracted from the input data 
(FORMAT 1) , the image data are down sampled. 

The down sampling algorithm can be performed in various 

35 ways that all have a different effect on the 
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performance of the algorithm. If down sampling by 
taking the average value in a neighborhood is used, a 
lot of document (small regions of color due to the 
printing process for e.g.) and scanner noise (jitter on 
5 the RGB signal) can be eliminated. 

More generally, an up/down sampling algorithm, which 
raise/reduce the image dimensions, can be used. 

The down sampled image data are then selected by means 
of a threshold, so obtaining an image having a pixels 
10 format without the spike noise (FORMAT 2) . In fact, the 
output data are considered ON if colorfulness of the 
input image pixel is above the threshold and OFF if it 
is lower than the threshold. 

The simplest case consists in applying a fixed 
15 threshold. The output of this stage is a down sampled 
version of the original image that has ON pixels in 
those regions where the color content of the original 
image were above the color threshold. Moreover, a rule 
for the colorfulness indicator can be considered. For 
20 example, evaluating the ON/OFF states of a particular 
pixel PX as well as the right RP and left pixel LP, the 
following rule can be applied: 

left pixel LP pixel PX right pixel RP result 

ON ON ON colorfull 

25 OFF OFF OFF no color. 

Moreover, different threshold values can be 
considerated with reference to different final devices. 

For example, a low resolution display does not need to 
receive a 16 billion color image data, since such 
30 device have no possibility of elaborating and 
displaying this kind of complex image data. An image 
data forwarded to a low resolution display can be 
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obtained by means of particular thresholding values by 
limiting the number of available colors, e.g. filtering 
pale colors and transforming then into white or 
"clustering" different type of "reds" in order to have 
5 only one "red" . 

An example of an acceptable range for the threshold 
values is 0 to 30 for an input image data of 0 to 255. 

The more complex case accumulates a histogram of the 
color content of the page and, using a heuristic, 
10 decides what the best threshold for the page is. 

The color information of single small groups of pixels 
can be further grouped together using known simple 
grouping techniques. The grouping step is performed on 
data of connected components, as shown in figure 14A 

15 (FORMAT 3) . This has the advantage of grouping regions 
of pixels that are considered colorful into bigger 
group. In this way, when the regions of colorful pixels 
are compressed (in a later stage) , not every pixel has 
to be compressed singularly. The compression of a 

20 larger group of pixels is more efficient than the 
singular compression of each single colorful region. 

The grouping of pixels also has the advantage of 
enabling the elimination of small groups of pixels that 
are still considered to be due to noise, or in any 
25 case, of insignificant size. 

The whole procedure that has been depicted can also be 
performed on a strip basis on the whole original image, 
without any modification, as shown in figure 14B. Strip 
based analysis produces a nice side effect on the 
30 grouping of pixels. In fact, if the grouping of pixels 
is performed on a strip basis, the grouping of pixels 
enables an approximation of the contour of colored 
regions, as shown in figure 15. 



20 



60980084-2 



Figure 14C shows a down sampling/thresholding/grouping 
device 10. The first component is an RGB to Chroma 
space converter 11. It converts, on a pixel by pixel 
basis, the color representation of every input pixel 
5 into a different color space representation. 

When the aim is making decisions on the colorfulness of 
a pixel, using the right color space representation is 
important . 

The RGB color space has been found not very convenient 
10 for this type of analysis. The color space used should 
have an indication of the colorfulness of the specific 
pixel. The HLN (Hue/Lightness/N??) color space was 
found particularly convenient and is used in the 
current real i zat ion . 

15 The indication of Chroma, in this HLN color space, is 
directly the content of the N channel, where N = max 
(R,G,B) - min (R,G,B) . 

The down sampling/thresholding/grouping device 10 
further comprises a down sampler 12, that down samples 
20 the N channel, and a thresholding device 13, in turn 
comprising a threshold selector 14 and a look-up- table 
LUT 15 which apply a threshold to the down sampled 
data . 

The output data is considered ON if colorfulness of the 

2 5 input image pixel is above the threshold and OFF if it 

is lower than the threshold. 

Moreover, the color information of single small groups 
of pixels can be further grouped together by means of a 
grouping block 16, using known simple grouping 

3 0 techniques on data of connected components, in order to 

improve the compression of the image data and eliminate 
small groups of pixels that are still considered to be 
due to noise, or in any case, of insignificant size. 
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It should be noted that a HLS to RGB converter (not 
shown in figure 14C) can also be added to the down 
sampling/ thresholding/grouping device 10 in order to 
obtain RGB output data. 

5 A simple grouping procedure called blobbing can be used 
in order to extract the images from a document, as 
shown in figure 16, where the blobbed regions should 
correspond to the images of the document . 

A further useful atomic operation on image data is the 
10 AND function, shown schematically in figure 17. In 
particular, after the recognition of the colorfulness 
content of each pixel and the blobbing of the image 
data, the singular images in the processed document can 
be separated by means of an AND function of such data 
15 and a mask, duly created on the basis of the following 
relationships : 

RGB AND WHITE = RGB 
RGB AND BLACK = BLACK. 

IMAGE 1 and IMAGE2 can be described as regions, 
20 identified by coordinates and dimensions. 

A sub-mask can be associated to each region to filter 
significant and insignificant data, as shown 
schematically in dotted lines in figure 16. 

Finally, another known atomic operation that can be 
25 used in the HLS space (or equivalent) is the filtering 
of achromatic portions, as shown in figure 18A, by 
means of a sub-cylinder section (ACHROMATIC) of the HLS 
color space. 

Figure 18B shows schematically a filter 17, according 
30 to the above filtering function. The filter 17 
comprises a RGB to HLS converter 18, connected to an 
HLS gray filter 19 and to a HLS to RGB converter 20. 
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According to the present invention, starting from a 
"flat" data sheet (the pixels 1 raster) a multi-layer 
representation is obtained. Moreover, using a dynamic 
pipeline, i.e. the collection of elementary functions 
5 and their links as above defined, output data having 
any kind of format are obtained. 

A configurable image processing device 21 according to 
the present invention comprises three major subsystems: 

1. the unit of processing called "transforms"; 

10 2. the unit of work called "packets"; 

3. the unit of transportation called "ports". 

With this three component, the device provide for 
transformations of image data, connections between the 
transforming units, and independent data 

15 representation . 

In particular, transforms process data in a context 
independent fashion, that being there is not implied 
knowledge coupled between transforms influencing the 
means of process data. Moreover transforms support 
20 multiple input and output capabilities through a port 
mechanism . 

In other words, transforms perform atomic operations on 
the data presented through a packet mechanism. 

Moreover, ports provide a communication protocol 
25 between processing units. The actual port protocol 
ranges from direct function call invocation, to 
operating system message queue/mail box protocols, to 
remote procedure invocations, as well as http links. 

Finally, packets provide the encapsulation of data. 
3 0 Data can represent actual image data in a variety of 
different formats, histogram information, compression 
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data, local grouping of data without requiring any 
impact to other components. By isolating the data into 
the packet mechanism data can be represented in the 
most efficient fashion for transportation or 
5 transformation without impacting either the 
transforming units or the means of data transportation. 

The configurable image processing device 21 and its 
three major subsystems isolate the means whereby data 
is transported around the system. By doing so, an image 
10 data processing pipeline is distributed in a 
transparent fashion, not influencing any other 
component within the image processing system. 

In fact, the configurable image processing device 21 
according to the present invention, select a particular 
15 group of atomic operations to be performed on the basis 
of the final device receiving the output data. By doing 
so, the device "create" a specific pipeline providing 
the required transformation of data. 

A functional and schematic representation of the 
20 configurable image processing device 21 is shown in 
figure 19A. The device 21 comprise a selector 22 of the 
particular group of atomic operations for a final 
device and a pipeline 23, that establishes the order 
and links between such atomic operations. 

25 In figure 19A is also shown a final reconstruction 
block 24, that process the results of the selector 22 
and the pipeline 23 in order to obtain the desired 
output layer of data. 

It should be noted that the pipeline 23 can also 
30 comprise sub-pipelines, in a more and more complicated 
structure, as shown in figure 19B. 

Advantageously, according to the present invention, 
each pipeline generates a layer, i.e. a specific data 
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organization. In this way, pipelines are created 
through a meta-data level of knowledge. Since a 
pipeline itself is an independent structure of 
different atomic operations and their links, there is 
5 no inherent knowledge incorporate in the hardware 
components of the image data processing device to 
convey the meaning/context of how the pipeline is to be 
applied. 

In other words, the effect of the pipeline is captured 
10 in how the atomic components are connected through 
ports and the data is transported and transformed. This 
meta- level of context is outside of the context 
implicitly provided by the components themselves; this 
task is left to the application that define and drives 
15 the dynamic pipeline. 

Advantageously, according to the dynamic pipeline as 
above defined, at any given moment there is never a 
static definition of the meaning of the pipeline 
itself. The context is represented in a meta-level 
20 definition that can be altered during run-time 
execution and can be reconfigured/rewired during the 
lifetime of the product even after the product has 
frozen "bits" and entered the marked. 

Finally, it is pointed out that a particularly useful 
25 application obtained from a dynamic pipeline according 
to the present invention deals with two particular 
layers of output data, i.e. the BW and color layers, 
needed in the increasely used PDF representation. 

A data processor 24 for obtaining processed color and 
30 BW layers is shown more precisely in figure 20. 

The data processor 24 has an input IN that receives the 
raster image data, for example in the RGB format, and 
is connected to a dual path layer creator component 25, 
which in turn outputs a first and a second layer, such 
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layers having different data compression rate. 

The first layer is inputted in a BW path 2 6 that 
outputs a processed BW layer. In a similar manner, the 
second layer is inputted in a color path 27 that 
5 outputs a processed color layer. The processed BW and 
color layers are finally inputted in a PDF device 28. 

More particularly, in the example shown in figure 20, 
the BW path 26 comprises a threshold block 29 connected 
in series to a compressor 30. A G4 compression is often 
10 used in the field of BW image data processing. 

Moreover, also in figure 20, the color path 27 
comprises a RGB to HLN converter 31, having the output 
N channel connected to a series of a down scale device 
32, an histogram and threshold selector 33, a look-up- 
15 table 34, a blob analysis block 35, a fill regions 
block 36 and a compressor 37. A JPEG compression is 
often used in the field of color image data processing. 

While it has been designed for embedded scanner 
applications, the configurable image processing device 
20 according to the present invention is platform 
independent as it can also operate on a personal 
computer or in a work station environment. 

There are several advantages of the image data 
processing method and device according to the present 

2 5 invention: 

1. The meta-level contextual knowledge used by the 
processing method and device is not hard coded into the 
physical implementation of the dynamic pipeline. 

2. The individual processing units/transforms as 

3 0 above defined can be realized in hardware, firmware, or 

software components to exploit capability of device. 

3. The transportation of data is de-coupled and 
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inherent transparent to the intent/purpose of the 
pipeline itself. 

4. The dynamic pipeline is completely self-contained, 
and its representation can be saved in whole or in 
part, so that context/state can be saved/replayed. 

5. The processing method leads to parallel, 
distributed solutions across any number of platform 
combinat ions . 

6. New transformations can be added after the product 
has been shipped through special code extending applet 
transform . 

7. The data representations can be shared on local 
copies of memory structures. 



